Papers by Lis Kanashiro Pereira
Attention-Focused Adversarial Training for Robust Temporal Reasoning (2022.lrec-1)
Copied to clipboard
| Challenge: | Current adversarial training approaches for NLP add adversarials to the embedding layer, ignoring other layers. |
| Approach: | They propose an enhanced adversarial training algorithm for fine-tuning transformer-based language models . they add the adversarials to multiple hidden states or attention representations of the model layers . |
| Outcome: | The proposed model improves performance on several temporal reasoning benchmarks and establishes new state-of-the-art results. |
Toward Building a Language Model for Understanding Temporal Commonsense (2022.aacl-srw)
Copied to clipboard
| Challenge: | Pre-trained language models such as BERT are still poor in temporal reasoning . commonsense reasoning is crucial for natural language processing (NLP) |
| Approach: | They propose to use multi-step fine-tuning and masked language modeling to predict mangled temporal indicators that are crucial for commonsense reasoning. |
| Outcome: | The proposed model improves performance on multiple time-related tasks. |